Hierarchical Temporal Memory (HTM) network systems represent a new approach to machine intelligence. In HTM network systems, training data comprising temporal sequences of spatial patterns are presented to a network of nodes. The HTM systems then build a model of the statistical structure inherent to the patterns and sequences in the training data, and thereby learns the underlying ‘causes’ of the temporal sequences and spatial patterns in the training data. The hierarchical structure of the HTM systems allows building of models that have very high dimensional input spaces using reasonable amounts of memory and processing capacity.
The HTM network is a group of processing nodes structured hierarchically. The nodes are hierarchically connected such that each node other than the bottommost node and the topmost node has several children nodes (i.e., nodes at a lower level) and one parent node (i.e., node at a higher level). Each level of nodes has a capacity to store and process information at different levels of abstraction. Bottom nodes are often coupled to receive sensory input pattern which has the lowest level of abstraction. The topmost node, on the other hand, produces an index indicating the cause of the sensory input pattern, and therefore, produces information at the highest level of abstraction. Each node between the bottom nodes and the top node is arranged to (i) propagate information “forward” (i.e., “up” an HTM hierarchy) to any connected parent node and/or (ii) propagate information “back” (i.e., “down an HTM hierarchy) to any connected children nodes.
The HTM training process is a form of unsupervised machine learning. However, during the training process, indexes representing the input patterns may be presented to the top node of the HTM network as well. These indexes allow the top node of the HTM network to associate particular categories with the underlying generative causes that are learned by the HTM network. Once an HTM network has built a model of a particular input space, it can be switched into an ‘inference’ mode. In this mode, novel input patterns are presented to the HTM network, and the HTM network will generate distributions that provide a quantitative measure of the degree of belief or likelihood that the input pattern was generated by the underlying cause associated with each of the categories to which the HTM network was exposed during the training stage.
For example, an HTM network might have been exposed to images of different animals, and simultaneously provided with category indexes such as ‘dog’, ‘cat’, and ‘bird’ that identifies objects in the images in the training mode. In the inference stage, the HTM network may be presented with a novel image of an animal, and the HTM network may generate distributions indicating the likelihood that the presented image was ‘dog’, ‘cat’ or ‘bird’. The distributions may include elements representing the relative belief or likelihood that the novel input pattern is an image of a ‘dog’, ‘cat’, ‘bird’, etc.